feat!: support v0.11.1#112
Merged
ILikeIneine merged 32 commits intomasterfrom Nov 14, 2025
Merged
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the codebase to support vllm v0.11.1, which involves significant refactoring around memory allocation, platform integration, and attention mechanisms. The changes appear to align with the goal of supporting the new vllm version. I have found one critical issue in the device allocator patch that could lead to a runtime error and have provided a fix.
Comment on lines
+48
to
+53
| if len(self._sleep_saved_buffers): | ||
| model = self.model_runner.model | ||
| for name, buffer in model.named_buffers(): | ||
| if name in self._sleep_saved_buffers: | ||
| buffer.data.copy_(self._sleep_saved_buffers[name].data) | ||
| self._sleep_saved_buffers = {} |
Contributor
There was a problem hiding this comment.
There is a potential AttributeError here. The self._sleep_saved_buffers attribute is only initialized within the sleep method, and only when level == 2. If wake_up is called after sleep(level=1) or before any call to sleep, self._sleep_saved_buffers will not exist on the object, causing a crash when len() is called on it.
To prevent this, you should safely check for the attribute's existence before trying to access it.
Suggested change
| if len(self._sleep_saved_buffers): | |
| model = self.model_runner.model | |
| for name, buffer in model.named_buffers(): | |
| if name in self._sleep_saved_buffers: | |
| buffer.data.copy_(self._sleep_saved_buffers[name].data) | |
| self._sleep_saved_buffers = {} | |
| if hasattr(self, "_sleep_saved_buffers") and self._sleep_saved_buffers: | |
| model = self.model_runner.model | |
| for name, buffer in model.named_buffers(): | |
| if name in self._sleep_saved_buffers: | |
| buffer.data.copy_(self._sleep_saved_buffers[name].data) | |
| self._sleep_saved_buffers = {} |
ILikeIneine
force-pushed
the
support-vllm-0.11.1
branch
from
a601543 to
1f48880
Compare
leex404
force-pushed
the
support-vllm-0.11.1
branch
2 times, most recently
from
ab31312 to
f516af8
Compare
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Xin Li <lixin1620@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
…l` (#115) * [fix] fix sample_recovered_tokens_kernel use too much private memory Signed-off-by: Xin Li <xin.li@metax-tech.com> * [fix] fix type error in bf16_paged_mqa_logits Signed-off-by: Xin Li <xin.li@metax-tech.com> * [chore] change file directory Signed-off-by: Xin Li <xin.li@metax-tech.com> --------- Signed-off-by: Xin Li <xin.li@metax-tech.com> Co-authored-by: Xin Li <xin.li@metax-tech.com> Signed-off-by: leex404 <lixin1620@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
ILikeIneine
force-pushed
the
support-vllm-0.11.1
branch
from
de238f9 to
32d2d83
Compare
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
related: vllm-project/vllm/pull/27322 Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
Signed-off-by: Hank <hcc.mayday@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
Signed-off-by: leex404 <lixin1620@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
This PR is for supporting vllm v0.11.1
Test Plan
Test Result
(Optional) Documentation Update
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.